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NON-LINEAR QUANTIZATION AND SIMILARITY MATCHING METHODS FOR 

RETRIEVING IMAGE DATA 

Description 

5 

Technical Field 

This invention relates to a method for retrieving 
image data; and, more particularly, to a method for 
10 constructing a bit-representation of an edge histogram 
descriptor having reduced bits and a method for retrieving 
image data by using the information effectively extracted 
from the coded representation of the descriptor, 

15 Background Art 

The Joint Photographic Experts Group (JPEG) is the 
international standard for still images and the Moving 
Picture Experts Group- 1 (MPEG-1), 2 are the standard for 

2 0 moving pictures. Regarding compressed image information, 
feature information for each image is extracted for 
applications such as extracting a key frame, image search, 
browsing or the like. 

To extract the feature information, brightness or 

25 color • histograms are widely used. The brightness and color 
histograms, respectively, represent relative frequency of 
brightness and color (red, green or blue) in an image. 
Especially, various methods of histogram comparison have 
been recently proposed for searching still images or 

30 digital videos that are stored digitally. As the 

histograms are used for image search and shot boundary 
detection, conventional histograms are to be improved. 
That is, it is necessary to adopt a histogram descriptor 
such as an edge histogram, which can represent the image 

35 content more efficiently. Also, the binary representation 
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of the descriptor should be compact and the computational 
complexity for the similarity matching should be low. 

A method of employing color histograms and edge maps 
for shot boundary detection is disclosed in U.S. Patent No. 
5 5,805,733 entitled "Method and system for detecting scenes 
and summarizing video sequences". Though the method is 
effective in that color information relative to a human 
visual system is extracted, the method does not include 
extraction of brightness information. 

10 A method in which color information is received and' 

then indexed by measuring the similarity of images using 
histogram intersection technique is disclosed in an article 
by M. J. Swain, et al., "Color Indexing", International 
Journal of Computer Vision, Vol. 7-1, pp. 11-32, 1991. 

15 However, this method does not use brightness and edge 
information and thus accuracy is not guaranteed. Also, 
since the histograms are generated using a discrete 
quantization technique in the conventional methods, a 
relatively large number of histogram bins are necessary to 

2 0 make equal performance. Consequently, inefficiency in 
storage and similarity measurement results. In addition, 
because feature extraction is conventionally performed in 
terms of pixels, there is a problem in that feature 
information is restrictively generated. 

2 5 In the meantime, as the histogram is widely used for 

image searching, etc., recently, an efficient way of 
storing histogram information is required. According to a 
conventional way to store the histogram, a histogram bin 
value is stored in a storage field of fixed size by 

30 normalization using linear quantization. Consequently, 
this method of linear quantization to the histogram storage 
causes a problem as the quantity of bits is increased. 
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Disclosure of Invention 

It is an object of the present invention to provide a 
method for constructing a database having image information 

5 representing a plurality of images with fewer bits to be 
stored in the database. 

It is another object of the present invention to 
provide a method for retrieving corresponding images in 
response to a query image based on a database with a high 
10 retrieval speed and accuracy. 

In accordance with one aspect of the present invention, 
there is provided a method for constructing a database 
having image information representing a plurality of images, 
the method comprising the steps of: a) calculating Lx5 

15 number of normalized edge histogram bins to generate L 
number of edge histograms of a target image, wherein each 
edge histogram has five normalized edge histogram bins and 
represents a spatial distribution of five reference edges 
in a sub-image and L is a positive integer, wherein the 

20 reference edges include four directional edges and a non- 
directional edge; b) non-linearly quantizing the Lx5 
number of normalized edge histogram bins to generate Lx5 
number of quantization index values for the target image; 
c) storing the Lx5 number of quantization index values to 

25 the database; and d) repeating the steps a) to c ) until all 
of the stored images are processed to thereby construct the 
database having the image information. 

In accordance with another- aspect of the present 
invention, there is provided a method for retrieving 

30 corresponding images in response to a query image based on 
a database, the method comprising the steps of: a) 
calculating L number of local edge histograms of the query 
image as an image descriptor for the query image, wherein 
each local edge histogram represents a spatial distribution 

35 of five reference edges in a sub-image and L is a positive 
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integer, wherein the reference edges include four 
directional edges and a non-directional edge; b) extracting 
a plurality of image descriptors for target images based on 
image information from the database, wherein the image 
5 descriptor for each target image includes L number of local 
edge histograms for each target image; c) comparing the 
image descriptor for the query image with each image 
descriptor for the target images to generate a comparison 
result; and d) retrieving at least one target image similar 
10 to the query image based on the comparison results. 

Brief Description of the Drawings 

Other objects and aspects of the invention will become 
15 apparent from the following description of the embodiments 
with reference to the accompanying drawings, in which: 

Fig. 1 is a block diagram illustrating a process for 
constructing a database having a plurality of image 
descriptors for corresponding images in accordance with the 
20. present invention; 

Fig. 2 shows an explanatory diagram depicting an image 
having 16 sub-images to be represented by image 
descriptors; 

Figs. 3A to 3E illustrate five types of edges to be 
25 used for an edge determination process in accordance with 
the present invention; 

Fig. 4 is an explanatory diagram demonstrating an 
image-block partitioned into four sub-blocks, to each of 
which filter coefficients are assigned; 
30 Figs. 5A to 5E show image-blocks, wherein sub-blocks 

of each image-block are provided with corresponding filtei~ 
coefficients for five edges; 

Fig. 6 is an explanatory diagram illustrating an array 
of 30 edge histogram bins corresponding to an image; 
35 Fig. 7 is a graph showing the difference in the 
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retrieval performance between a non-linear quantization 
using a Lloyd-Max quantizer and a linear quantization; 

Figs. 8A and 8B depict two block diagrams, each 
showing a process for retrieving a desired image in 
5 response to an input of query image in accordance with the 
present invention; and 

Figs. 9A to 9C are explanatory diagrams demonstrating 
13 groups of sub-images provided for representing semi- 
global edge histograms in accordance with the present 
10 invention. 

Best Mode for Carrying Out the Invention 

Hereinafter, preferred embodiments in accordance with 

15 the present invention will be described in detail referring 
to the accompanying drawings . 

Referring to Fig. 1, there is shown a block diagram 
illustrating a process for constructing a plurality of 
image descriptors for corresponding images in accordance 

2 0 with the present invention. 

At the processing block S101, an image to be processed 
is inputted and divided into NxN sub-images, e.g., 4x4 sub- 
images, wherein N is a positive integer. The sub-images 
are then coupled to a processing block S103 where a first 

25 image descriptor for the image is extracted. That is, an 
edge histogram for each sub-image is obtained by using a 
multiplicity of edges and then 80 normalized local edge 
histogram bins for the image are coupled to the processing 
block S105 as the first image descriptor. 

30 At the processing block S105, the first image 

descriptor is non-linearly quantized to thereby generate a 
corresponding second image descriptor which is, e.g., a 
group of quantization index values. 

Thereafter, the second image descriptor for the image 

35 is inputted and stored in a database S107. The above 
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process is repeated until all of the images to be 
represented using the second descriptors are processed. 

Referring to Figs. 2 to 6 , there are shown explanatory 
diagrams for illustrating a process for obtaining the first 
5 image descriptor described referring to Fig. 1. 

As shown in Fig. 2, in order to obtain a corresponding 
first image descriptor , an inputted image 200 is divided 
into NxN non-overlapping sub-images. In the example shown, 
the image is divided into 4x4 sub-images, to thereby form 
10 16 rectangular sub-images 211 to 226. Each of the sub- 
images contains a plurality of pixels. 

As described hereinbefore, the first image descriptor 
represents normalized edge histogram bins for generating 
the edge histogram for each sub-image wherein the edge 
15 histogram represents the spatial distribution of edges in 
the sub-image. 

In order to extract the edge histogram, each sub-image 
is then divided into M x K non-overlapping rectangular 
image-blocks wherein the size of the image-blocks depends 
2 0 on the size of the image. Each image-block is used in an 
edge determination process, in which an image-block is 
described by using one of its edges. 

In accordance with one embodiment of the present 
invention, as shown in Figs. 3A to 3E, the edge 
25 determination process is provided with five edges, one of 
which is selected for an image-block. The edges can 
include various types of directional edges, preferably, 
vertical, horizontal, 45 degree and 135 degree edges 301, 
303 , 305 and 307; and a non-directional edge 309 including 
30 at least one edge of undesignated direction. 

In order to generate an edge histogram for a sub-image, 
it is necessary to detect an edge feature from an image- 
block. That is, the edge determination process is 
performed in order to determine which one of the edges can 
35 be assigned to an image- block. The extraction can be 
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performed using a method applying digital filters in the 
spatial domain. 

In the edge determination process, as shown in Fig. 4, 
an image-block is partitioned into four sub-blocks. That 
5 is, as shown, a reference numeral 400 denotes an image- 
block and reference numerals 411, 413, 415 and 417 denote 
sub-blocks also labeled 0, 1, 2 and 3, respectively, for 
the image-block 400, wherein a corresponding filter 
coefficient is assigned to each sub-block to be able to 
10 obtain a set of edge magnitudes . 

In accordance with one embodiment of the present 
invention, each image-block 400 is partitioned into 2x2 
sub-blocks each of which is labeled 0, 1, 2 or 3 . 

For each image block, a set of five edge magnitudes 
15 corresponding to five types of edges are obtained by using 
the following equations: 

«„(*,/) -| g«*(U)*/v<*) I E< 3- 1 

m ;i (i,/H ^ k (i 9 j)xf h (k) | Eq. 2 

2 0 m d _ l35 (i, /) - | g a k 0\ J) x/.-i35 (*) | E< 3 • 4 

m nd (i, j) - | g fl t (i, /) (*) | Ec 3 * 5 

where m v {ij) , m h (i 9 j) , m d „ 45 (i 9 j) , m d _ 12S (i, j) and m nd (i,j) , 
respectively, denote vertical, horizontal, 45 degree, 135 
degree and non-directional edge magnitudes for a (/,/) fch 
25 image-block; a k (i,j) denotes an average gray level for a sub- 
block labeled k in the (i, j) th image-block and f v (k), f h (k), 

/ d -4ff(*) ' / rf -i3s(*) and denote ' respectively, filter 

coefficients for the vertical, horizontal, 45 degree, 135 
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degree and non-directional edges where k = 0, 1, 2 and 3, 
each representing a number labeling each sub-block. 

Referring to Figs. 5A to 5E, there are shown filter 
coefficients for each edge. 
5 As shown, reference numerals 501 , 503, 505 , 507 and 

509 show filter coefficients for the vertical, horizontal, 
45 degree, 135 degree and non-directional edges, 
respectively. Each image-block can be represented by using 
a selected one among five edge magnitudes, wherein each 

10 magnitude is calculated for respective edges. 

In order to determine an edge corresponding to an 
image-block, the five edge magnitudes obtained by using the 
above equations are compared with each other. According to 
the comparison, the image-block is expressed by the one 

15 edge having a maximum edge magnitude relative to the others, 
where the maximum edge magnitude should also be greater 
than a predetermined threshold value. If the maximum edge 
magnitude is less than the predetermined threshold value, 
it is determined that the image-block contains no edge. 

20 When the selected edge for the image-block is 

determined as a result of the edge magnitude comparison, a 
corresponding edge histogram bin for the sub-image is 
increased by one. There are five types of edge histogram 
bins, i.e., vertical, horizontal, 45 degree, 135 degree and 

25 non-directional edge histogram bins. The five edge 
histogram bins are components for representing the edge 
histogram. The detection of corresponding edges for all of 
the image-blocks included in a sub-image is performed and 
then an edge histogram bin corresponding to each detected 

30 edge is increased by one to thereby generate an edge 
histogram, called a local edge histogram, for the sub-image. 
The detection of edges and generation of edge histogram are 
performed for all of the 16 sub-images. 

The local edge histogram represents the distribution 

35 of the five types of edges in a sub-image, i.e., an edge 

S 
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histogram for a sub-image. Since the number of sub-images 
is fixed at 16 and each sub-image is assigned five edge 
histogram bins, 80 edge histogram bins are needed to 
generate corresponding local edge histograms for all of the 
5 16 sub-images. That is, semantics of each bin of BinCounts 
is defined as shown in Table 1: 



Table 1 







BinCounts [ 0 ] 


Vertical edges m sud- image \ u , u ) 


BinCounts [ 1 ] 


Horizontal edges in suD-image (u,u; 


BinCounts [ 2 ] 


4 5 degree edges m sud- image ( u , u) 


BinCounts [ 3 ] 


135 degree edges in sub-image (0,0) 


T*> -i -r*\ f> /—v i i J— Y A ~\ 

oinLounus L ^ J 


Nnn-Hi rprtional edcres in sub— imacre 
(0,0) 


BinCounts [ 5] 


Vertical edges in sub-image (0,1) 






BinCounts [ 74 ] 


Non-directional edges in sub-image 
(3,2) 


BinCounts [ 75 ] 


Vertical edges in sub-image (3,3) 


BinCounts [76] 


Horizontal edges in sub-image (3,3) 


BinCounts [ 77 ] 


45 degree edges in sub-image (3,3) 


BinCounts [ 78 ] 


135 degree edges in sub-image (3,3) 


BinCounts [ 79 ] 


Non-directional edges in sub-image 
(3,3) 



10 where BinCount[0], BinCount[l] . • . BinCount[79] 

represent final coded bits for the edge histogram 
descriptor. 

Referring to Fig. 6, there is shown an exemplary array 

•of 80 edge histogram bins corresponding to an image. 

15 For example, a local edge histogram for a sub- image 

211 at (0,0) of the image 200 shown in Fig. 2 includes 

vertical, horizontal, 45 degree, 135 degree and non- 

9 
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directional edge histogram bins 600, 601, 602, 603 and 604, 
respectively, as shown in Fig. 6. In the same way, a local 
edge histogram for a sub-image 212 at (0,1) in Fig. 2 
includes five edge histogram bins 605, 606, 607 , 608 and 
5 609 in the same sequence as that of bins for the sub-image 
211. Consequently, a total of 80 edge histogram bins are 
needed to generate respective 16 local edge histograms for 
all of the 16 sub-images, wherein the total 80 bins are 
calculated by multiplying five edge histogram bins by 16 

10 sub-images. 

In order to obtain a first image descriptor, each edge 
histogram bin in a local edge histogram for a sub-image is 
normalized by dividing each bin by the total number of the 
image-blocks included in the sub-image. Thereby, each edge 

15 histogram bin for the local edge histogram has a bin value 
ranging from 0 to 1 . The normalized bin values for the 
image are then coupled to the processing block S105 shown 
in Fig. 1 as the first image descriptor. At the processing 
block SI 05, the first image descriptor is non-linearly 

2 0 quantized by using a number of quantization tables. 

That is, in order to obtain a second image descriptor, 
the normalized bin values are quantized to obtain binary 
representations thereof. The quantization should be 

performed for the 80 normalized bin values. In this case, 

25 the normalized bin values are non-linearly quantized to be 
able to minimize the overall number of bits for the binary 
representations . 

As a result, a group of quantization index values is 
obtained as the second image descriptor. The non-linear 

30 quantization is performed using a non-linear quantizer 
designed with, e.g., a Lloyd-Max algorithm in accordance 
with one embodiment of the present invention. 

In order to perform the quantization, five non-linear 
quantization tables for each of vertical edge, horizontal 

35 edge, 45 degree edge, 135 degree edge and non-directional 
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edge histogram bins are used therein, which are 
representatively set forth in Tables 2 to 6 : 

Table 2: Quantization Table for the Vertical Edge 
Histogram Bin 



Tndpx f ^bits/bin} 


Range 


Representative 
value 


0 


0 . 0000000-0 .0343910 


0.010867 


1 


0 . 0343910-0 . 0787205 


0.057915 


2 


0.0787205-0.1221875 


0 . 099526 


3 


0 . 1221875-0 . 1702110 


0 . 144849 


4 


0 . 1702110-0 .2280385 


0 . 195573 


5 


0.2280385-0.3092675 


0.260504 


6 


0 . 3092675-0 .4440795 


0 . 358031 


7 


0.4440795-1.0000000 


0.530128 


Table 3: Quantization Table for the Horizontal Edge 
Histogram 3in 


Index ( 3bits/bin) 


Range 


Representative 
value 


0 


0.0000000 - 0.0411000 


0.012266 


1 


0.0411000 - 0.0979065 


0.069934 


2 


0 . 0979065 - 0 . 1540930 


0 . 125879 


3 


0.1540930 - 0.2128515 


0 .182307 


4 


0.2128515 - 0.2789795 


0.243396 


5 


0.2789795 - 0.3631455 


0 .314563 


6 


0 . 3631455 - 0 .4880235 


0 .411728 


7 


0 .4880235 - 1 . 0000000 


0.564319 



10 

Table 4: Quantization Table for the 45 Degree Edge 
Histogram Bin 
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index ( 3bits /bin ) 


Range 


Representative 
value 


n 
u 


0.0000000 - 0.0150225 


0.004193 


i 
± 


0.0150255 - 0.0363560 


0.025852 


o 


0.0363560 - 0.0576895 


0.046860 


3 


u. uj / ooyo — u . UoUyuz j 


n n ^ p ^ i q 

U • UDO OX!? 


4 


0.0809025 - 0.1083880 


0 . 093286 


5 


0.1083880 ~ 0.1424975 


0 . 123490 


6 


0.1424975 ~ 0.1952325 


0. 161505 


7 


0.1952325 ~ 1-0000000 


0.228960 



Table 5: Quantization Table for the 135 Degree Edge 
Histogram Bin 

5 



Index( 3bits/bin) 


Range 


Representative 
value 


0 


0 . 0000000-0 .0150490 


0 . 004174 


1 


0 . 015 04 90-0 .03607 80 


0 . 025924 


2 


0 . 0360780-0 . 0566975 


0. 046232 


3 


0 . 0566975-0 . 0784090 


0.067163 


4 


0.0784090-0.1025230 


0.089655 


5 


0 . 1025230-0. 1336475 


0. 115391 


6 


0. 1336475-0. 1848245 


0 . 151904 


7 


0 . 1848245-1 . 0000000 


0 .217745 



Table 6: Quantization Table for the Non-Directional 
Edge Histogram Bin 



Index ( 3bits/bin) 


Range 


Representative 
value 


0 

I 


0 . 0000000-0 . 0292225 


0 . 006773 



12 



WO 02/339 7S 



I'CT/KRO I/O 1754 



I 


0.0292225-0.0801585 


0 . 051667 


2 


0 .0801585-0 . 1374535 


0 . 103650 


3 


0 .1374535-0 . 1952415 


0. 166257 


4 


0 . 1952415-0 . 2549585 


0.224226 


5 


0.2549585-0. 3210330 


0 .285691 


6 


0. 3210330-0.4036735 


0 .356375 


7 


0.4036735-1.0000000 


0.450972 



where the optimal number of bits per bin is fixed at 
three in order to have eight quantization levels in the 
above quantization tables in accordance with present 
5 invention. The second image descriptor is then stored in 
the database S107 and is retrieved in response to an input 
of a query image. 

Referring to Fig. 7, there is shown a graph depicting 
the difference in retrieval performance between a non- 
10 linear quantization using a Lloyd-Max quantizer and a 
linear quantization. 

As shown, the Y axis of the graph represents an 
average normalized modified retrieval rank (ANMRR) 
indicating the retrieval performance, and the X axis 
15 represents the number of bits per image based on the number 
of bits per edge bin in generating the quantization tables. 
A solid line 7 00 and a broken line 701 show retrieval 
* performances when using the non-linear quantization and the 
linear quantization, respectively. 
20 As can be seen from the solid line 700, if the number 

of bits per edge bin is three, the number of bits per image 
is 24 0 , obtained by multiplying three bits by 80 edge 
histogram bins. The ANMRR takes a real number between 0 
and 1. As the ANMRR is closer to 0, the retrieval 
25 performance, i.e., retrieval accuracy, is better. 
Accordingly, it can be readily appreciated that assignment 
of three bits to each edge histogram bin leads to the most 
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efficient encoding in consideration of both retrieval 

performance and bit generation quantity. 

Referring to Figs. 8A and 8B, there are shown block 

diagrams illustrating two methods for retrieving a desired 
5 image in response to an input of' query image in accordance 

with a preferred embodiment of the present invention* 

If a query image is received, the query image is 

processed in the same manner of the processing blocks S101 

and S103 shown in Fig. 1. That is, a first image 
10 descriptor for the query image is obtained in the above 

described manner and includes normalized edge histogram 

bins for the query image. 

Thereafter, local edge histograms, a global edge 

histogram and semi-global edge histograms for the query 
15 image are generated based on the normalized edge histogram 

bins as an image descriptor. The global edge histogram 

represents the edge distribution for the whole image space. 

The global edge histogram and the semi-global histograms 

will be described hereinafter in more detail. 
20 On the other hand, according to Fig. 8A, there is 

shown a first method for retrieving a desired image in 

response to an input of a query image by using a number of 

non-linear inverse quantization tables in accordance with a 

preferred embodiment of the present invention, wherein the 
25 non-linear inverse tables can be of Tables 2, 3, 4, 5 and 6 

as described above. 

When a query image is inputted, the same process as 

that in the processing block S101, i.e., the image division 

process is performed at a processing block S801. 
30 At a processing block S803, the same process as that 

in the processing block S103, i.e., the edge histogram 

generation, is performed. 

In order to achieve a high retrieval performance, a 

global edge histogram and semi-global edge histograms for 
35 the query image can be further generated at a processing 
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block S805 , based on the local edge histogram bins that are 
generated at the processing block S803. 

For a data matching process, a number of second image 
descriptors for target images are retrieved sequentially 
5 from the pre-established database S107. For a stored 
target image, a group of quantization index values are 
retrieved and coupled to the non-linear inverse- 
quantization tables S807 . Through the use of the non- 
linear inverse-quantization tables, the quantization index 

10 values are then converted into normalized edge histogram 
bins for the target image. 

At a processing block S809, the normalized edge 
histogram bins are then used in extracting local edge 
histograms, a global edge histogram and semi-global edge 

15 histograms for the target image. That is, in order to 
achieve a high retrieval performance, the local edge 
histograms having the normalized edge histogram bins, the 
global edge histogram and the semi-global edge histograms 
are used in the data matching process as an image 

2 0 descriptor for a target image. In the data matching 
process S811, by calculating a distance between the local, 
semi-global and global edge histograms of the query image A 
and the target image B, a similarity between the two images 
is determined as follows: 

25 

79 4 

Distanc<A5) = g| Local_A\i]-Local_B\i] ! + 5x ^| Global_A\i]-Global_B\i] | 

64 

+ ^| Semi_Global_A[i]- Semi_Globa!_B\i] \ 

Eq . 6 

where Local_A[i] and Local_B[i] denote, respectively, 
index values assigned to i th bins of each of local edge 
30 histograms of the images A and B; Global_A[ ] and Globai_3[] 
denote, respectively, index values assigned to i" h bins of 
each of global edge histograms of the images A and B; and 
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Semi_Global_A[ ] and Semi__Global_3 [ ] denote, respectively, 
index values assigned to i~ h bins of each of semi-global 
edge histograms of the images A and 3. Since the number of 
bins for the global edge histogram is relatively smaller 
5 than that of the local and semi-global edge histograms, a 
weighting factor of five is applied in the above equation. 

As explained above, using equation 6, the similarity 
between the two images A and B can be measured by referring 
to the inverse quantization tables. In this case, since 
10 edge histogram bin values for the image should be decoded 
by referring to the inverse quantization tables, equation 6 
is generally used in applications for complicated but 
accurate retrieval. Herein, each of the inverse 

quantization tables corresponds to each of the edge 
15 quantization tables shown in Tables 2 to 6 . 

The above procedure is then repeated until all of the 
target images are processed. 

On the other hand, according to Fig. 8B, there is 
shown a second method for retrieving a desired image in 
2 0 response to an input of query image without using a number 
of non-linear inverse quantization tables in accordance 
with a preferred embodiment of the present invention. That 
is, in the second method, the local edge histograms for an 
image can be restored at a high speed without the above 
25 inverse quantization tables. 

For an inputted query image, the processing blocks 
S801, S803 and S805 are performed as described hereinbefore. 

At a processing block S808, a set of quantization 
index values are retrieved 'from the database S107 and 
30 normalized to thereby generate a set of normalized index 
values. The set of normalized index values' is then coupled 
to a processing block S810. 

At a processing block S810, the normalized index 
values are used as values of edge histogram bins for the 
35 stored image. That is, a global edge histogram and semi- 
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global edge histograms for the stored image are generated 
based on the values of normalized edge histogram bins, 
called local edge histogram bin values of the decoded image. 
In a processing block S812, by calculating a distance 
5 between the image descriptor having the local, semi-global 
and global edge histograms of the query image and those of 
the decoded image, a similarity between the two images, 
i.e., the query image A and the target image B, is 
determined . 

10 Since there is a monotonic increment of the edge 

histogram bin values in the non-linear quantization tables, 
each of the quantization tables in accordance with the 
present invention has a linearly incremental characteristic 
in some sections around 0 where an edge occurrence 

15 probability is high. This means that a set of index values 
itself can be used as a similarity measurement value in the 
image retrieval process without referring to the inverse 
quantization tables . 

The following equation 7 is applied to obtain the 

2 0 distance of edge histograms between two images A and B 
obtained directly based on the index values for the stored 
image as follows: 



79 



Distanc<A,£) - g| Local_A\i]-Local_B\i] | +5x Global_A\i]-Global_B\i] \ 

+ g| Semi_Global_A\i]-Semi_Global_B\i] \ 
25 Eq. 7 

Using equation 7, similarity can also be measured by 
using a set of encoded index values itself in consideration 
of a partially linear characteristic of the inverse 
30 quantization table. Therefore, it can be seen that a 
retrieval process described in Fig. 33 is faster than that 
described in Fig. 8A. 
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Referring to Figs. 9A to 9C there are shown exemplary 
diagram illustrating 13 groups of sub-images provided for 
semi-global edge histograms in accordance with a preferred 
embodiment of the present invention. 
5 In setting up image regions for edge detection, as 

shown in Fig. 2, the image is divided vertically and 
horizontally into four equal parts to form 16 sub-images, 
and one local edge histogram for each sub-image is 
generated. Also, because 16 sub-images can always be 
10 formed irrespective of the image size, the quantization 
index values of the image can be encoded in a fixed length 
of bits and stored in the database. 

The local edge histograms encoded from each sub-image 
can be merged and reused to thereby obtain additional edge 
15 histograms of a global edge histogram and semi-global edge 
histograms . 

The global edge histogram is generated by merging the 
local edge histogram for each of 16 sub-images in an image. 
As aforementioned, each edge histogram has five edge 

20 histogram bins, i.e., the vertical, horizontal, 45 degree, 
135 degree and non-directional edge histogram bins. To 
merge 16 local edge histograms, it is necessary to add the 
16 vertical edge histogram bins from each edge histogram 
together. In the same way, the summation is performed for 

25 horizontal, 45 degree, 135 degree and non-directional edge 
histogram bins. Accordingly, the edge distribution 

information for the whole image can be acquired from the 
global edge histogram. 

Referring again to Figs. 9A to 9C, four sets of four 

30 sub-images, each set included in each of columns 1 to 4 in 
vertical direction, are, in sequence, set to 1 st to 4 th sets 
of the sub-images 901, 903, 905 and 907 and four sets of 
four sub-images, each set included in each of rows 5 to 8 
in horizontal direction , are, in sequence, set to 5 th to 8~ h 

35 sets of four sub-images 909, 911, 913 and 915, respectively. 



3NS OCC I D. < WO , C2335 7 8 A \ „ \ „ 



WO 02/33978 



PCT/KKU 1/0 1754 



Also, five sets of four sub-images are set to 9~", I0^ h , 11" , 
12 th and 13 th sets of sub-images 917, 919, 921, 923 and 925. 
Each of 9 th to 12 th sets 917, 919, 921 and 923, respectively, 
has a sub-image located on the left-top, on the right-top, 
5 on the left-bottom and on the right-bottom of the image and 
the 13 th set 92 5 has four sub-images around the center of 
the image. The 1 st to 13 th sets of sub-images 901 to 925 
include 65 edge histogram bins which are obtained by 
multiplying 13, denoting the total number of the sub-images, 
10 by five, denoting the number of edge histogram bins for 
each sub-image. Consequently, there are generated semi- 
global edge histograms having 65 edge histogram bins for 
the 13 sets of four sub-images. 

By using the 65 edge histogram bins for the semi- 
15 global edge histograms and five edge histogram bins for the 
global edge histogram as well as the 80 edge histogram bins 
for the local edge histogram in order to determine 
similarity between two images, the retrieval efficiency of 
the system is improved. 
20 Especially, by selecting the above 13 sets of four 

sub-images to generate the semi-global edge histograms, the 
retrieval performance can be improved because the 13 sets 
have a characteristic less sensitive to a location change 
of objects in the image. 
25 In accordance with the present invention, the number 

of bits necessary for storing the quantization index values 
can be greatly reduced. Furthermore, the complexity of the 
similarity calculation can be significantly decreased by 
using the non-linear quantizer. 
3 0 Although the preferred embodiments of the invention 

have been disclosed for illustrative purpose, those skilled 
in the art will appreciate that various modifications, 
additions, and substitutions are -possible, without 
departing from the scope and sprit of the invention as 
35 disclosed in the accompanying claims. 
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Claims 

1 . A method for constructing a database having image 
information representing a plurality of images, the method 

5 comprising the steps of: 

a) calculating Lx5 number of normalized edge histogram 
bins to generate L number of edge histograms of a target 
image, wherein said each edge histogram has five normalized 
edge histogram bins and represents a spatial distribution 

10 of five reference edges in a sub-image and Lisa positive 
integer, wherein the reference edges include four 
directional edges and a non-directional edge; 

b) non-linearly quantizing the Lx5 number of 
normalized edge histogram bins to generate Lx5 number of 

15 quantization index values for the target image; 

c) storing the Lx5 number of quantization index values 
to the database; and 

d) repeating the steps a) to c) until all of the 
stored imaged are processed to thereby construct the 

2 0 database having the image information. 

2. The method as recited in claim 1, wherein the 
directional edges include a vertical edge, a horizontal 
edge, a 45 degree edge and a 135 degree edge, and the non- 
25 directional edge represents an edge of undesignated 

direction excluding the four directional edges. 

3. The method as recited in claim 2, wherein the 
vertical edge, the horizontal edge, the 45 degree edge, the 

30 135 degree edge and the non-directional edge are, 
respectively, expressed as: 
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edge 



fionzontai 
ed»e 



4:> degree 
edge 



13d degree 
edge 



non-directional 
edge 



4. The method as recited in claim 3, wherein the step 
a) includes the steps of: 
5 al) partitioning the target image into L number of 

sub-images, wherein each sub-image is further partitioned 
into S x K number of image-blocks, S and K being positive 
integers; and 

a2 ) assigning one of the reference edges to each 
10 image-block to thereby generate L number of edge histograms 
for the target image, wherein each edge histogram includes 
the Lx5 edge histogram bins; and 

a3 ) normalizing the edge histogram bins by S x K which 
is the number of the image-blocks contained in one sub- 
15 image, to thereby generate Lx5 number of normalized edge 
histogram bins, each normalized edge histogram bin having 
a real number between 0 and 1 • 



5. The method as recited in claim 4, wherein the step 
20 al) includes the steps of: 

al-1) dividing the target image into N x N non- 
overlapping sub-images to thereby form L number of 
rectangular sub-images, N being a positive integer; and 

al-2) dividing each sub-image into S x K non- 
25 overlapping blocks to thereby form S x K number of 
rectangular image-blocks . 

6. The method as recited in claim 5, wherein the step 
a2 ) includes the steps of: 

30 a2-l) assigning one of the reference edges to each 

image block; and 
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a2-2 ) counting the number of each reference edge 
included in each sub-image to generate the L number of the 
edge histograms for the target image. 

5 7. The method as recited in claim 6, wherein the step 

a2-l) includes the steps of: 

a2-ll) dividing each image-block into 2x2 sub- 
blocks ; 

a2-12) assigning a corresponding filter coefficient to 
10 each sub-block; 

a2-13) calculating a set of five edge magnitudes 
corresponding to five edges for each image-block by using 
the filter coefficient; and 

a2-14) expressing the image-block as an edge having a 
15 maximum edge magnitude by comparing the calculated edge 
magnitudes to each other, 

8. The method as recited in claim 7, wherein the five 
edge magnitudes are obtained by using five equations, which 
20 are expressed as: 

3 

'"v(*\;)H g ^ (*) | ; 

3 

3 
3 

m d-i3s 0'> J) - | ^ a k (*, 7) x/ rf -i35 (*) I ; and 

3 

2 5 m nd (i, D = | g a * & -0 (*) i ' whe r e m„ (i ? ;) , m h (L j) , m d _ A5 (/, ;) , 

m d-izs(Uj) and m nd (ij) , respectively, denote vertical, 
horizontal, 45 degree, 135 degree and non-directional edge 
magnitudes for a (z, ;) th image-block; a k {i, j) denotes an 
average gray level for a sub-block assigned k in the (i 9 j)* h 
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image-block and f v (k), f b (k) , f d . 4S (k) , f d . l35 (k) and f nd {k) denote, 
respectively, filter coefficients for the vertical, 
horizontal, 45 degree, 135 degree and non-directional edges, 
where k represents a number assigned to each sub-block. 

5 

9. The method as recited in claim 7, wherein the image 
block is expressed as containing no edge when the maximum 
edge magnitude is less than a predetermined threshold value. 

10 10. The method as recited in claim 1, wherein the step 

b) includes the step of bl ) non-linearly quantizing each of 
the normalized edge histogram bins by using a corresponding 
non-linear quantization table, wherein the step bl) is 
provided with five non-linear quantization tables, each 

15 corresponding to a particular reference edge. 

11. The method as recited in claim 10, wherein the 
step bl) includes the steps of: 

b2-l) mapping each of the normalized edge histogram 
20 bins into a representative value contained in each 
corresponding reference edge quantization table; and 

b2-2) generating three bits of a quantization index 
value representing a representative value for each 
normalized edge histogram bin to thereby generate L x 5 
25 number of the quantization index values as a second image 
descriptor for the target image. 

12. The method as recited in claim 10, wherein the 
normalized edge histogram bins are non-linearly quantized 

30 using a non-linear quantizer based on a Lloyd-Max algorithm. 

13. The method as recited in claim 5, wherein said N 

is 4 . 



35 



14. The method as recited in claim 12, wherein the 
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quantization table includes a table for a vertical edge 
histogram bin representing the number of vertical edges in 
the sub-image, which is expressed as: 



I ndex 


Range 


Representative value 


0 


0.0000000 — 0.014^910 


0.010867 


1 


0.0343910 - 0. 0787205 


0 . 057915 


2 


0. 0787205 - 0 . 1221875 


0.099526 


3 


0. 1221875 - 0. 1702110 


0 . 144849 


4 


0. 1702110 - 0.2280385 


0. 195573 


5 


0.2280385 ~ 0.3092675 


0.260504 


6 


0.3092675 ~ 0.4440795 


0. 358031 


7 


0.4440795 ~ 1.0000000 


0.530128 



15- The method as recited in claim 12, wherein the 
quantization table further includes a table for a 
horizontal edge histogram bin representing the number of 
horizontal edges in the sub-image, which is expressed as: 



Index 


Range 


Representative 
value 


0 


0 


. 0000000 




0. 


0411000 


0.012266 


1 


0 


. 0411000 




0. 


0979065 


0 . 069934 


2 


0 


. 0979065 




0. 


1540930 


0 . 125879 


3 


0 


. 1540930 




0. 


2128515 


0 . 182307 


4 


0 


.2128515 




0. 


2789795 


0.243396 


5 


0 


.2789795 




0 . 


3631455 


0 .314563 


6 


0 


.3631455 




0 . 


4880235 


0 .411728 


7 


0 


.4880235 




1 . 


0000000 


0 . 564319 
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16. The method as recited in claim 12, wherein the 
quantization table further includes a table for a 45 degree 
edge histogram bin representing the number of 45 degree 
edges in the sub-image, which is expressed as: 



Index 


Range 


XX \3 KJ -L C O w 11 U CA -L. V w V d _L LI v3 


0 


0.0000000 ~ 0.0150225 


0 . 004193 


1 


0.0150255 - 0.0363560 


0 . 025852 


2 


0.0363560 - 0.0576895 


0 . 046860 


3 


0.0576895 ~ 0.0809025 


0 . 068519 


4 


0 . 0809025 - 0 . 1083880 


0.093286 


5 


0 . 1083880 - 0 . 1424975 


0 . 123490 


6 


0 . 1424975 - 0 . 1952325 


0 . 161505 


7 


0.1952325 - 1.0000000 


0.228960 



17. The method as recited in claim 12, wherein the 

quantization table further includes a table for a 135 

10 degree edge histogram bin representing the number of 135 
degree edges in the sub-image, which is expressed as: 



Index 


Range 


Representative value 


0 


0.0000000 - 0.0150490 


0 . 004174 


1 


0.0150490 ~ 0.0360780 


0 . 025924 


2 


0.0360780 ~ 0.0566975 


0 . 046232 


3 


0.0566975 - 0.0784090 


0 . 067163 


4 


0 . 0784090 ~ 0 . 1025230 


0 . 089655 


5 


0 . 1025230 - 0 . 1336475 


0 .115391 


6 


0 . 1336475 - 0 . 1848245 


0. 151904 


7 


0 . 1848245 ~ 1 . 0000000 


0.217745 
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18. The method as recited in claim 12, wherein the 
quantization table further includes a table for a non- 
directional edge histogram bin representing the number of 
non-directional edges, which is expressed as: 



Index 


Range 


Representative value 


0 


0.0000000 - 0.0292225 


0.006778 


1 


0.0292225 - 0.0801585 


0.051667 


2 


0.0801585 ~ 0.1374535 


0 . 108650 


3 


0.1374535 ~ 0.1952415 


0. 166257 


4 


0. 1952415 - 0 .2549585 


0.224226 


5 


0.2549585 ~ 0.3210330 


0.285691 


6 


0.3210330 - 0.4036735 


0.356375 


7 


0.4036735 ~ 1.0000000 


0.450972 



19. The method as recited in claim 6, wherein 
semantics of said each edge histogram bin of BinCounts is 
defined as: 

0 







BinCounts [ 0 ] 




BinCounts [ 1 ] 


Horizontal edges in sub-image (0,0) 


BinCounts [2 ] 


45 degree edges in sub-image (0,0) 


BinCounts [ 3 ] 


135 degree edges in sub-image (0,0) 


BinCounts [ 4 ] 


Non-directional edges in sub-image 
(0,0) 


BinCounts [ 5 ] 


Vertical edges in sub-image (0,1) 


• 




BinCounts [ 74 ] 


Non-directional edges in sub-image 
(3,2) 


BinCounts [ 75 ] 


Vertical edges in sub-image (3,3) 


BinCounts [ 76 ] 


Horizontal edges in sub-image (3,3) 
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BinCounts [ 7 / ] 


45 degree edges in sub-image (3,3) 


BinCounts [78] 


135 degree edges in sub-image (3,3) 


BinCounts [ 7 9 ] 


Non-directional edges in sub-image 
(3,3) 



20. A method for retrieving corresponding images in 
response to a query image based on a database, the method 
comprising the steps of: 

5 a) calculating L number of local edge histograms of 

the query image as an image descriptor for the query image, 
wherein each local edge histogram represents a spatial 
distribution of five reference edges in a sub-image and L 
is a positive integer, wherein the reference edges include 

10 four directional edges and a non-directional edge; 

b) extracting a plurality of image descriptors for 
target images based on image information from the database, 
wherein the image descriptor for each target image includes 
L number of local edge histograms for each target image; 

15 c) comparing the image descriptor for the query image 

with each image descriptor for the target images to 
generate a comparison result; and 

d) retrieving at least one target image similar to the 
query image based on the comparison result. 

20 

21. The method as recited in claim 20, wherein each 
local edge histogram has five edge histogram bins 
corresponding to the reference edges. 

25 22. The method as recited in claim 20, wherein the 

directional edges include a vertical edge, a horizontal 
edge, a 45 degree edge and a 135 degree edge, and the non- 
directional edge represents an edge of undesignated 
direction excluding the four directional edges. 

30 

23. The method as recited in claim 22, wherein the 
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step a) includes the steps of: 

al) partitioning the query image into N x N non- 
overlapping sub-images to thereby form L number of 
rectangular sub-images, wherein each sub-image is further 
5 partitioned into S x K number of image-blocks, S and K 
being positive integers; and 

a2) assigning one of the reference edges to - each 
image-block to thereby generate L number of edge histograms 
for the query image, wherein the edge histogram includes 
10 the L x 5 edge histogram bins; and 

a3) normalizing the edge histogram bins by S x K which 
is the number of image-blocks contained in one sub-image, 
to thereby generate L x 5 number of normalized edge 
histogram bins, each normalized edge histogram bin having 
15 a real number between 0 and 1. 

24. The method as recited in claim 23, wherein the 
step a2 ) includes the steps of: 

a2-l) assigning one of the reference edges to each 
2 0 image block; and 

a2-2) counting the number of each reference edge 
included in each sub-image to generate the L number of the 
edge histograms for the query image. 

25 25. The method as recited in claim 17, wherein the 

step a2-l) includes the steps of: 

a2-ll) dividing each image-block into 2x2 sub- 
blocks ; 

a2-12) assigning a corresponding filter coefficient to 
30 each sub-block; 

a2-13) calculating a set of five edge magnitudes 
corresponding to five edges for each image-block by using 
the filter coefficient; and 

a2-14) expressing the image-block as an edge having a 
35 maximum edge magnitude by comparing the calculated edge 
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magnitudes to each other. 

26. The method as recited in claim 25, wherein the 
five edge- magnitudes are obtained by using five equations, 
which are expressed as: 

3 



m v (z, ;) - | g a, (/, ;) x/ v (*) J ? 

3 

w " & = i ^ a * & # W I ; 



w^Ci,;)-! ^ I? 

3 

m d-l35 (h D - I g (*, 7) x/'^i35 (*) 1 5 and 

3 

10 1 */,*(*) I * where « v (/,y) , m h (i 9 j) , m d ^(i 9 j) , 

Wd-us&i) and m nd(}>D ' respectively, denote - vertical, 
horizontal, 45 degree, 135 degree and non-directional edge 
magnitudes for a th image-block; a k (i,j) denotes an 

average gray level for a sub-block assigned k in the (i,j) th 
15 image-block, and f v (k) , f k (k) , f d . A5 (k) , f d _ l35 (k) and f nd (k) 
denote, respectively, filter coefficients for the vertical, 
horizontal, 4 5 degree, 135 degree and non-directional edges, 
where k represents a number assigned to each sub-block. 

20 27. The method as recited in claim 25, wherein the 

image block is expressed as containing no edge when the 
maximum edge magnitude is less than a predetermined 
threshold value. 

25 28 . The method as recited in claim 20, wherein the 

image descriptors for the query and target images further 
include a global edge histogram and R number of the semi- 
global histograms based on the Lx5 number of normalized 
edge histogram bins, respectively, with R being a positive 
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integer . 

29. The method as recited in claim 28, wherein the 
global edge histogram represents an edge distribution in a 
5 whole space of the query and target images and each semi- 
global edge histogram represents an edge distribution in a 
corresponding set of sub-images of the query and target 
images . 

10 30. The method as recited in claim 28, wherein said N 

and R are 4 and 13, respectively. 

31. The method as recited in claim 30, wherein each of 
the 13 semi-global edge histograms is generated for each of 

15 13 sets of four sub-images, the 13 sets including four sets 
of four sub-images, each set including four sub-images in 
each of first to fourth columns of the image in a vertical 
direction; four sets of four sub-images, each set including 
four sub-images in each of first to fourth rows in a 

20 horizontal direction; four sets of four sub-images, each 
set including a corresponding sub-image and three sub- 
images neighboring the corresponding sub-image, wherein the 
corresponding sub-image is respectively located on a left- 
top, on a right-top, on a left-bottom and on a right-bottom 

25 of the image; and a set including four sub-images around a 
center of the image. > 

32. The method as recited in claim 20, wherein the 
step b) includes the steps of: 

30 bl) retrieving L x 5 number of quantization index 

values for each of the target images ; 

b2 ) converting each of the L x 5 number of 
quantization index values into L x 5 number of normalized 
edge histogram bins for each target image by using five 

35 non-linear inverse quantization tables; and 

30 
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b3) generating L number of local edge histograms 
based on the L x 5 number of normalized edge histogram bins, 

33. The method as recited in claim 32, wherein the 
5 step b) further includes the step of: b4 ) further- 
generating a global edge histogram and R number of semi- 
global histograms for each of the target images based on 
the L x 5 number of normalized edge histogram bins. 

10 34 . The method as recited in claim 20, wherein the 

step b) includes the steps of: 

bl) retrieving L x 5 number of quantization index 
values for each of the target images; 

b2 ) converting each of the L x 5 number of 
15 quantization index values into L x 5 number of normalized 
edge histogram bins for each target image by normalizing 
the L x 5 number of quantization index values; and 

b3) generating L number of local histograms based on 
the L x 5 number of normalized edge histogram bins. 

20 

35. The method as recited in claim 34, wherein the 
step b) further includes the step of: b4 ) further 
generating a global edge histogram and R number of semi- 
global histograms for each of the target images based on 

25 the L x 5 number of normalized edge histogram bins. 

36. The method as recited in claim 33, wherein the 
step c) includes the step of: 

estimating a distance between the query image and said 
30 each target image by an equation as: 

79 ^ r 

DistanciAB) = Jj| Local _A[i\- Local _B\i] l + 5x ^! Global_A\i]-Global_B\i\ \ 

64 

+ ^| Semi_Global_A\i]-Semi_Global_B[i\ \ 
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where Local_A[ i ] and Local__B[i] denote, respectively, 
the edge histogram bins of BinCount[i] of the query image A 
and the target image B; Global_A[ ] and Global_B[] denote, 
respectively, the edge histogram bins for the global edge 
5 histograms of the query image A and the target image B; and 
Semi_j31obal_A[ ] and Semi_Global_B [ ] denote, respectively, 
the histogram bin values for the semi-global edge histogram 
bins of the query image A and the target image B. 

10 37. The method as recited in claim 35, wherein the 

step c) includes the step of: 

estimating a distance between the query image and said 
each target image by an equation as: 



where Local_A[i] and Local_B[i] denote, respectively, 
the edge histogram bins of BinCount[i] of the query image A 
and the target image B; Global_A[ ] and Global__B[] denote, 
20 respectively, the edge histogram bins for the global edge 
histograms of the quex-y image A and the target image B; and 
Semi_Global_A[ ] and Semi_Global_B [ ] denote, respectively, 
the histogram bin values for the semi-global edge histogram 
bins of the query image A and the target image B. 



38. A method for constructing a bit-representation of 
an edge histogram descriptor for a target image, comprising 
the steps of: 

a) partitioning the target image into 16 sub-images, 
30 wherein each sub-image is further partitioned into S x K 
number of image-blocks, S and K being positive integers; 



15 




25 



b) assigning one of five reference e 



dges 



to 



each 
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image-block to thereby generate 16 edge histograms for the 
target image, wherein the edge histogram includes the 80 
edge histogram bins and the reference edges include four 
directional edges and a non-directional edge; 
5 c) normalizing the edge histogram bins contained in 

each edge histogram by S x K to thereby generate 8 0 
normalized edge histogram bins, each normalized edge 
histogram bin having a real number between 0 and 1; and 

d) non-linearly quantizing 80 normalized edge 

10 histogram bins to generate 80 quantization index values as 
a second image descriptor for the target image. 

39. The method as recited in claim 38, wherein each 
quantization index value is represented by three bits. 

15 

40. The method as recited in claim 39, wherein the 
directional edges include a vertical edge, a horizontal 
edge, a 45 degree edge and a 135 degree edge, and the non- 
directional edge represents an edge of undesignated 

2 0 direction excluding the four directional edges. 
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